강화학습을 이용한 KSTAR 토카막 운전 궤적 설계

학위논문(박사) -- 서울대학교대학원 : 공과대학 에너지시스템공학부, 2022. 8. 나용수.토카막에서 정밀한 물리 실험을 하기 위해서는 먼저 특정한 내부 플라즈마 상태를 달성하고 유지하는 것이 필요하다. 특히 상용 핵융합로 운전을 위해서는 자기유체역학적으로 안정적인 플라즈마 영역 내에서의 제어가 필수적이며, 고출력의 핵융합 반응을 일으킬 수 있는 플라즈마 상태를 유지할 수 있어야 한다. 기존에는 실험에서 목표로 하는 플라즈마 상태를 달성하기 위해, 다양한 토카막 운전 조건에서의 사전 시뮬레이션과 실험에서의 추가적인 시행착오가 필요하였다. 이 경우 많은 인적 노동력과 시간이 소요되었으며, 새로운 목표 상태들에 대해 매번 동일한 수준의 시행착오가 요구된다는 문제가 있다. 본 논문은 목표 플라즈마 상태를 달성하기 위한 토카막의 운전 경로를 설계하는 기계학습 기반의 알고리즘 개발을 다룬다. 해당 알고리즘은 기존의 상당한 시뮬레이션 및 시행착오를 수행하는 작업들을 대체할 수 있으며, 이를 통해 보다 빠르고 효율적으로 가능성 높은 운전 조건을 도출할 수 있다. 첫 번째로, 토카막 운전 설계 인공지능 모델의 훈련 환경에 해당하는 토카막 시뮬레이션 기술이 연구되었다. KSTAR 실험 데이터의 패턴을 학습하여 시간에 따른 플라즈마 상태를 순차적으로 예측하는 LSTM 기반의 인공신경망 모델을 개발하였다. 데이터 학습 과정에서 과적합 및 오차 누적 등의 문제를 해결하기 위해 다양한 수치적 기법들이 적용되었다. 학습된 모델은 KSTAR의 다양한 운전 시나리오 방전들에 대해 우수한 예측 정확도를 보여주었으며, 신뢰도 분석을 통해 모델이 과적합되지 않음을 확인하였다. 또한 해당 모델을 기반으로, 실시간 상호작용을 통한 가상 토카막 실험이 가능하도록 그래픽 사용자 인터페이스 (GUI)를 개발하였다. 해당 GUI 상에서 사용자가 토카막 운전 변수들을 조정함에 따라 플라즈마의 변화를 실시간으로 시각적으로 확인할 수 있기 때문에 물리 연구 뿐 아니라 전문가 교육용으로서의 의의가 있다. 두 번째로, 개발된 시뮬레이터 상에서 스스로 운전 변수들을 조정하여 목표로 하는 플라즈마 상태를 달성하는 인공지능 모델을 강화학습 기법을 이용하여 훈련하였다. 이를 통해 목표 플라즈마 상태를 달성하기 위한 적절한 토카막 운전 경로를 설계하는 알고리즘을 개발할 수 있다. 먼저 목표 β_N 달성을 위해 플라즈마 전류, 플라즈마 형태 및 가열 파워를 결정하는 모델을 훈련하였다. 훈련된 모델이 설계한 운전 경로를 이용하여 실험을 수행해본 결과 오차범위 내에서 목표 β_N이 도출됨을 검증하였다. 특히 한정된 가열 조건에서 높은 성능을 달성하기 위해 플라즈마 형태를 적절히 조정하여 가둠 성능을 향상시키는 것을 확인하였다. 이후 보다 더 구체적인 플라즈마 상태를 달성하기 위해, 플라즈마 압력 (β_p) 뿐 아니라 자기장 구조 (q_95) 및 내부 인덕턴스 (l_i)의 다중 파라미터들의 목표값을 동시에 달성케 하는 인공지능 모델 또한 훈련하였다. 해당 모델이 설계한 운전 경로를 실제 실험에 적용해본 결과, 다중 플라즈마 파라미터들이 성공적으로 목표값으로 제어됨을 확인하였다. 본 논문에서 개발된 기계학습 기반 알고리즘은 추후 고성능 운전 시나리오 연구에 도움을 줄 수 있으며, 정밀한 물리 조건을 요구하는 실험에서 초기 조건 달성을 위한 기술로 적용될 수 있을 것으로 기대된다. 추후 실시간 피드백 제어에 적용됨으로써 다양한 상황에서 자율적으로 제어되는 핵융합로 기술 개발을 위한 초석이 될 수 있을 것으로 전망한다.In order to conduct a sophisticated physics experiment in a tokamak, it is necessary to achieve and sustain a specific target plasma state first. Especially, the commercial fusion reactor requires controlling plasmas within a stable parametric range and maintaining a favorable plasma state for high fusion power generation. Conventionally, we had to conduct numerous simulations with various tokamak operating conditions and experiment with trials and errors for achieving a target plasma state. This takes lots of labor and time and requires the same level of trial and error for different targets each time. This thesis addresses the development of a reinforcement learning (RL)-based algorithm that designs the tokamak operation trajectory to achieve a given target plasma state. This algorithm replaces the conventional manual tasks of numerous simulative experiments and provides a probable tokamak operation condition faster and more efficiently. First, the tokamak simulator, corresponding to the training environment of the RL agent that designs the operation trajectory, was developed. An LSTM-based neural network was trained that sequentially predicts the plasma state over time by learning the patterns of the KSTAR experimental data. Various numerical techniques were applied to prevent overfitting and error accumulation during the training process. The trained model showed reasonable prediction accuracy for various operation scenarios in KSTAR, and reliability analyses verified that the model was not significantly overfitted. Furthermore, based on the trained model, we developed a graphical user interface (GUI) to enable virtual tokamak experiments through real-time interaction. By adjusting the tokamak operation parameters on the GUI, the user can visually check the plasma evolution in real time, which can be useful not only for physics research but also for expert education. Second, an artificial agent was trained using a reinforcement learning technique, that adjusts the operation parameters to achieve a target plasma state in the developed simulator. This agent can design a plausible tokamak operation trajectory to achieve a given target after training. First, the agent was trained to determine the plasma current, the plasma shape, and the heating power to achieve the target β_N. We conducted a KSTAR experiment with the operation trajectory designed by the trained agent, and it was verified that the target performance was achieved within the tolerance range. In particular, it was observed that the confinement enhancement factor was improved by adjusting the plasma shape to achieve high performance under limited heating conditions. Moreover, in order to achieve a more specific plasma state, another RL agent was trained to achieve multiple targets of β_p, q_95, and l_i simultaneously. The KSTAR experiment with the RL operation design showed that multiple plasma parameters were successfully controlled to the target values. The RL-based algorithm addressed in this thesis can provide clues for the research of advanced operation scenarios and can be applied to achieve initial plasma states in experiments that require sophisticated physical conditions. By applying this algorithm to real-time feedback control in the future, it will become a basis for developing a self-operating fusion reactor that can be autonomously controlled to achieve high power generation.1. Introduction １ 1.1. Advanced operation scenario in tokamak ３ 1.2. Machine learning in fusion research ４ 1.2.1. Precedent research ４ 1.2.2. AI operation trajectory design and control ６ 1.2.3. Simulation of tokamak plasmas １０ 1.3. Objective and outline of this dissertation １５ 2. Data-driven tokamak simulator １７ 2.1. Predictive modeling with DNN １８ 2.1.1. Construction and training of the LSTM-based model １８ 2.1.2. Demonstration and validation ２９ 2.2. Analysis for model reliability ３２ 2.2.1. Uncertainties in dataset ３２ 2.2.2. Consistency with prior knowledge ３４ 3. Operation trajectory design algorithm ３６ 3.1. Environment of the RL training ３７ 3.2. Control of normalized beta ３９ 3.2.1. The action, observation, and reward for RL ３９ 3.2.2. RL training ４４ 3.3. Simultaneous control of multiple parameters ４５ 3.3.1. The action, observation, and reward for RL ４５ 3.3.2. RL training ４９ 4. Validation in KSTAR ５３ 4.1. Control of normalized beta ５４ 4.1.1. Case 1: β_N control to 2.4 and 1.8 ５４ 4.1.2. Case 2: β_N control to 2.7 ５７ 4.1.3. Case 3: β_N control to 3.5 ６０ 4.2. Simultaneous control of multiple 0D parameters ６２ 4.2.1. Experiment with RL-designed trajectory ６２ 4.2.2. Comparison with other shots in dataset ６６ 5. Conclusion ６９ Bibliography ７１ Abstract in Korean ７７박

Large-scale Text-to-Image Generation Models for Visual Artists' Creative Works

Large-scale Text-to-image Generation Models (LTGMs) (e.g., DALL-E), self-supervised deep learning models trained on a huge dataset, have demonstrated the capacity for generating high-quality open-domain images from multi-modal input. Although they can even produce anthropomorphized versions of objects and animals, combine irrelevant concepts in reasonable ways, and give variation to any user-provided images, we witnessed such rapid technological advancement left many visual artists disoriented in leveraging LTGMs more actively in their creative works. Our goal in this work is to understand how visual artists would adopt LTGMs to support their creative works. To this end, we conducted an interview study as well as a systematic literature review of 72 system/application papers for a thorough examination. A total of 28 visual artists covering 35 distinct visual art domains acknowledged LTGMs' versatile roles with high usability to support creative works in automating the creation process (i.e., automation), expanding their ideas (i.e., exploration), and facilitating or arbitrating in communication (i.e., mediation). We conclude by providing four design guidelines that future researchers can refer to in making intelligent user interfaces using LTGMs.Comment: 15 pages, 3 figure

A Unified Compression Framework for Efficient Speech-Driven Talking-Face Generation

Virtual humans have gained considerable attention in numerous industries, e.g., entertainment and e-commerce. As a core technology, synthesizing photorealistic face frames from target speech and facial identity has been actively studied with generative adversarial networks. Despite remarkable results of modern talking-face generation models, they often entail high computational burdens, which limit their efficient deployment. This study aims to develop a lightweight model for speech-driven talking-face synthesis. We build a compact generator by removing the residual blocks and reducing the channel width from Wav2Lip, a popular talking-face generator. We also present a knowledge distillation scheme to stably yet effectively train the small-capacity generator without adversarial learning. We reduce the number of parameters and MACs by 28$\times$ while retaining the performance of the original model. Moreover, to alleviate a severe performance drop when converting the whole generator to INT8 precision, we adopt a selective quantization method that uses FP16 for the quantization-sensitive layers and INT8 for the other layers. Using this mixed precision, we achieve up to a 19$\times$ speedup on edge GPUs without noticeably compromising the generation quality.Comment: MLSys Workshop on On-Device Intelligence, 2023; Demo: https://huggingface.co/spaces/nota-ai/compressed_wav2li

South Koreas Soft Power in the Era of the Covid-19 Pandemic: An Analysis of the Expert Survey in Europe

Since the recent outbreak of the Covid-19, South Korea hasdemonstrated successful pandemic management that can beexemplary to other countries. This paper analyzes how SouthKoreas responses to the Covid-19 pandemic has changed theperceptions of the country in Europe. Through a survey conducted with Korea experts in 16 European countries, this paper documents the positive recognition of South Koreas pandemic management by the European public. Part of the positive appraisal can be attributed to South Koreas extensive testing, high technology, and the culture of wearing a face mask, while the opinions were more mixedregarding its comprehensive tracking and tracing strategy due to privacy concerns. Furthermore, the findings of the survey show that Europeans overall perception of South Korea has improved together with its Covid-19 management. This evidence suggests that thecountrys success in pandemic management can be an instrumentof public diplomacy to enhance its soft power, for which thegovernment of South Korea currently invests considerable efforts.We are grateful to the KDI School of Public Policy and Management for providing financial support. We also thank the Korea experts who participated in the survey

Characteristics of Classified Aerosol Types in South Korea during the MAPS-Seoul Campaign

During the Megacity Air Pollution Studies-Seoul (MAPS-Seoul) campaign from May to June 2015, aerosol optical properties in Korea were obtained based on the AERONET sunphotometer measurement at five sites (Anmyon, Gangneung_WNU, Gosan_SNU, Hankuk_UFS, and Yonsei_University). Using this dataset, we examine regional aerosol types by applying a number of known aerosol classification methods. We thoroughly utilize five different methods to categorize the regional aerosol types and evaluate the results from each method by inter-comparison. The differences and similarities among the results are also discussed, contingent upon the usage of AERONET inversion products, such as the single scattering albedo. Despite several small differences, all five methods suggest the same general features in terms of the regionally dominant aerosol type: Fine-mode aerosols with highly absorbing radiative properties dominate at HankukUFS and Yonsei_University; non-absorbing fine-mode particles form a large portion of the aerosol at Gosan_SNU; and coarse-mode particles cause some effects at Anmyon. The analysis of 3-day back-trajectories is also performed to determine the relationship between classified types at each site and the regional transport pattern. In particular, the spatiotemporally short-scale transport appears to have a large influence on the local aerosol properties. As a result, we find that the domestic emission in Korea significantly contributes to the high dominance of radiation-absorbing aerosols in the Seoul metropolitan area and the air-mass transport from China largely affects the western coastal sites, such as Anmyon and Gosan_SNU

The implication of the air quality pattern in South Korea after the COVID-19 outbreak

By using multiple satellite measurements, the changes of the aerosol optical depth (AOD) and nitrogen dioxide (NO2) over South Korea were investigated from January to March 2020 to evaluate the COVID-19 effect on the regional air quality. The NO2 decrease in South Korea was found but not significant, which indicates the effects of spontaneous social distancing under the maintenance of ordinary life. The AODs in 2020 were normally high in January, but they became lower starting from February. Since the atmosphere over Eastern Asia was unusually stagnant in January and February 2020, the AOD decrease in February 2020 clearly reveals the positive effect of the COVID-19. Considering the insignificant NO2 decrease in South Korea and the relatively long lifetime of aerosols, the AOD decrease in South Korea may be more attributed to the improvement of the air quality in neighboring countries. In March, regional atmosphere became well mixed and ventilated over South Korea, contributing to large enhancement of air quality. While the social activity was reduced after the COVID-19 outbreak, the regional meteorology should be also examined significantly to avoid the biased evaluation of the social impact on the change of the regional air quality

Exploring evidence of positive selection reveals genetic basis of meat quality traits in Berkshire pigs through whole genome sequencing

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.Abstract Background Natural and artificial selection following domestication has led to the existence of more than a hundred pig breeds, as well as incredible variation in phenotypic traits. Berkshire pigs are regarded as having superior meat quality compared to other breeds. As the meat production industry seeks selective breeding approaches to improve profitable traits such as meat quality, information about genetic determinants of these traits is in high demand. However, most of the studies have been performed using trained sensory panel analysis without investigating the underlying genetic factors. Here we investigate the relationship between genomic composition and this phenotypic trait by scanning for signatures of positive selection in whole-genome sequencing data. Results We generated genomes of 10 Berkshire pigs at a total of 100.6 coverage depth, using the Illumina Hiseq2000 platform. Along with the genomes of 11 Landrace and 13 Yorkshire pigs, we identified genomic variants of 18.9 million SNVs and 3.4 million Indels in the mapped regions. We identified several associated genes related to lipid metabolism, intramuscular fatty acid deposition, and muscle fiber type which attribute to pork quality (TG, FABP1, AKIRIN2, GLP2R, TGFBR3, JPH3, ICAM2, and ERN1) by applying between population statistical tests (XP-EHH and XP-CLR). A statistical enrichment test was also conducted to detect breed specific genetic variation. In addition, de novo short sequence read assembly strategy identified several candidate genes (SLC25A14, IGF1, PI4KA, CACNA1A) as also contributing to lipid metabolism. Conclusions Results revealed several candidate genes involved in Berkshire meat quality; most of these genes are involved in lipid metabolism and intramuscular fat deposition. These results can provide a basis for future research on the genomic characteristics of Berkshire pigs